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April  30,  2010 


Congressional  Committees 


Subject:  P?'eliminary  Observations  on  the  Department  of  Defense’s 
Countemarcotics  Performance  Measurement  System 


The  Department  of  Defense  (DOD)  leads  detection  and  monitoring  of  aerial  and 
maritime  transit  of  illegal  drugs  into  the  United  States  in  support  of  law  enforcement 
agencies.  Additionally,  DOD’s  counternarcotics  activities  include  sharing  information 
with  U.S.  and  foreign  agencies,  as  well  as  helping  foreign  countries  build  their 
countemarcotics  capacity.  In  support  of  these  activities,  Congress  provided  just  over 
$6.1  billion  to  DOD’s  Countemarcotics  Central  Transfer  Account  from  fiscal  year 
2005  through  2010. 


In  November  2005,  GAO  recommended  that  DOD,  in  conjunction  with  other  agencies 
performing  countemarcotics  activities,  develop  and  coordinate  countemarcotics 
performance  measures.  GAO  has  found  that  measuring  performance  allows 
organizations  to  track  the  progress  they  are  making  toward  their  goals  and  provides 
managers  a  basis  for  making  key  decisions  to  improve  programs  and  results.  In  May 
2007,  the  Office  of  National  Drug  Control  Policy  (ONDCP)  released  a  circular  that 
required  agencies,  including  DOD,  to  submit  a  report  on  their  performance-related 
information  for  countemarcotics  activities,  including  how  the  agency’s  performance 
measures  are  used  in  the  management  of  the  program. 


The  National  Defense  Authorization  Act  for  Fiscal  Year  2010  (Pub.  L.  No.  111-84,  § 
1016)  mandated  that  GAO  report  on  the  performance  evaluation  system  used  by  DOD 
to  assess  its  countemarcotics  activities.  To  satisfy  this  mandate,  in  April  2010,  we 
briefed  congressional  staff  from  the  defense  committees.  This  letter  summarizes  the 
preliminary  observations  conveyed  during  those  briefings.  The  briefing  slides  used  to 
provide  the  briefing  are  attached  as  enclosure  I. 


We  plan  to  issue  the  final  results  of  our  evaluation  of  DOD’s  countemarcotics 
performance  evaluation  system  in  a  report  to  be  released  in  the  summer  of  2010  that 
will  focus  on  the  extent  to  which:  (1)  DOD’s  countemarcotics  performance  measures 
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enable  DOD  to  track  progress  toward  achieving  its  goals  and  (2)  DOD  uses 
performance  information  to  support  its  counternarcotics  mission. 


In  summary,  the  briefing  provided  to  congressional  staff  noted  that: 


•  DOD  has  developed  performance  measures  for  its  countemarcotics  activities. 
DOD  has  also  developed  a  database  to  collect  performance  information, 
including  measures,  targets,  and  results.  However,  these  performance 
measures  lack  a  number  of  the  attributes,  such  as  being  clearly  stated  and 
having  measurable  targets,  which  GAO  considers  key  to  successful 
performance  measures. 


•  DOD  is  currently  reassessing  its  performance  measures  for  its 

countemarcotics  activities.  DOD  hired  a  contractor  to  assist  the  department  in 
revising  its  guidance  for  the  development  of  countemarcotics  performance 
measures.  DOD  officials,  however,  noted  that  the  department  still  faces 
challenges  measuring  the  performance  of  its  countemarcotics  activities.  These 
challenges  include  creating  performance  measures  that  assess  program 
outcomes;  ensuring  adequate  resources,  such  as  expertise  in  performance 
management,  are  available  to  develop  performance  measures;  and  ensuring 
that  the  data  used  to  measure  DOD  performance  is  reliable. 


•  It  is  unclear  the  extent  to  which  DOD  uses  the  performance  information  it 
collects  through  its  database  to  manage  its  countemarcotics  activities. 
Previous  work  by  GAO  has  found  that  the  full  benefit  of  collecting 
performance  information  is  realized  only  when  managers  use  this  information 
to  inform  key  decisions  to  improve  programs  and  results,  such  as  identifying 
corrective  actions,  allocating  resources,  and  sharing  best  practices  for 
program  implementation.  However,  officials  at  U.S.  Central  Command,  U.S. 
Southern  Command,  and  the  Joint  Interagency  Task  Force-South  told  us  they 
rarely  use  the  database  to  identify  performance  problems  or  manage 
countemarcotics  activities.  Rather,  they  tend  to  manage  programs  using 
measures  and  data  not  submitted  to  the  database.  We  plan  to  examine  this 
issue  in  more  detail  in  our  report  to  be  released  in  the  summer  of  2010. 


We  conducted  this  performance  audit  from  December  2009  to  April  2010,  in 
accordance  with  generally  accepted  government  auditing  standards.  Those  standards 
require  that  we  plan  and  perform  the  audit  to  obtain  sufficient,  appropriate  evidence 
to  provide  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our  audit 
objectives.  We  believe  that  the  evidence  obtained  provides  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives. 


We  provided  drafts  of  this  report  to  DOD  and  ONDCP  for  comment.  DOD  provided  no 
comments.  ONDCP  provided  technical  comments,  which  we  have  incorporated 
where  appropriate. 
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We  are  sending  copies  of  this  report  to  interested  congressional  committees,  the 
Secretary  of  Defense,  and  the  Director  of  the  Office  of  National  Drug  Control  Policy. 
The  report  also  is  available  at  no  charge  on  GAO’s  Web  site  at  http://www.gao.gov. 

If  you  or  your  staff  members  have  any  questions  about  this  information,  please 
contact  me  at  (202)  512-4268  or  fordj@gao.gov.  Contact  points  for  our  Offices  of 
Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last  page  of  this 
report.  GAO  staff  who  made  major  contributions  to  this  report  are  listed  in  enclosure 


II. 


Jess  T.  Ford 

Director,  International  Affairs  and  Trade 


Enclosures 
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List  of  Congressional  Committees 

The  Honorable  Carl  Levin 
Chairman 

The  Honorable  John  McCain 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Daniel  K.  Inouye 
Chairman 

The  Honorable  Thad  Cochran 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Ike  Skelton 
Chairman 

The  Honorable  Howard  P.  McKeon 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Norman  D.  Dicks 

Chairman 

The  Honorable 

Ranking  Member  C.W.  Bill  Young 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Enclosure  I:  Briefing  for  Congressional  Committees 
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Objectives,  Scope,  and  Methodology 


•  The  National  Defense  Authorization  Act  of  2010  (Pub.  L.  No. 
1 1 1  -84,  §  1016)  mandated  that  GAO  report  on  the 
performance  evaluation  system  used  by  the  DOD  to  assess 
its  counternarcotics  activities. 


•  This  briefing  contains  information  on  DOD’s 

counternarcotics  performance  evaluation  system,  including 
the  following: 

•  the  extent  to  which  DOD’s  counternarcotics  performance 
measures  enable  DOD  to  track  progress  toward 
achieving  its  goals,  and 

•  preliminary  observations  on  DOD’s  use  of  performance 
information  to  support  its  counternarcotics  mission. 
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Objectives,  Scope,  and  Methodology 


•  To  address  these  objectives  we 

•  Interviewed  officials  from  the  Office  of  National  Drug 
Control  Policy  (ONDCP)  and  DOD  components  including 
the  following: 

•  Office  of  the  Deputy  Assistant  Secretary  of  Defense  for 
Counternarcotics  and  Global  Threats  (DASD-CN), 

•  U.S.  Central  Command  (CENTCOM), 

•  U.S.  Southern  Command  (SOUTHCOM), 

•  Joint  Interagency  Task  Force-South  (JIATF-S),  and 

•  DOD  Inspector  General  (DOD-IG). 
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Objectives,  Scope,  and  Methodology 


•  To  address  these  objectives  we 

•  reviewed  and  analyzed  DOD  strategy  and  budget 
documents,  as  well  as  DOD  guidance  on  performance 
measures,  and 

•  evaluated  a  generalizable  random  sample  of  DOD’s 
unclassified  fiscal  year  2009  counternarcotics 
performance  measures  (71  of  171  measures)  to  assess 
the  extent  to  which  these  measures  adhered  to  GAO 
criteria  on  the  key  attributes  of  performance  measures 
(see  table  I).1 


1  Prior  GAO  work  identified  nine  attributes  of  successful  performance  measures.  See  GAO,  Tax  Administration:  IRS  Needs  to 
Further  Refine  Its  Tax  Filing  Season  Performance  Measures,  GAO-03-143  (Washington,  D.C.:  Nov.  22,  2002). 
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Objectives,  Scope,  and  Methodology 


•  We  conducted  this  performance  audit  from  December  2009 
to  April  2010,  in  accordance  with  generally  accepted 
government  auditing  standards.  Those  standards  require  that 
we  plan  and  perform  the  audit  to  obtain  sufficient,  appropriate 
evidence  to  provide  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives. 

•  After  conducting  additional  audit  work,  we  plan  to  issue  the 
final  results  of  our  evaluation  of  DOD’s  counternarcotics 
performance  measurement  system  in  a  report  to  be  released 
in  the  summer  of  201 0. 
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Background 


•  The  DOD  leads  detection  and  monitoring  of  aerial  and 
maritime  transit  of  illegal  drugs  into  the  United  States  in 
support  of  law  enforcement  agencies. 


To  accomplish  this  mission,  DOD  has  identified  three  goals 

•  detecting  and  monitoring  drug  trafficking, 

•  sharing  information  with  U.S.  and  foreign  agencies,  and 


n  countries  build  their  counternarcotics 


•  In  support  of  these  activities,  Congress  provided  just  over 
$6.1  billion  to  DOD’s  Counternarcotics  Central  Transfer 
Account  from  fiscal  year  2005  to  fiscal  year  201 0. 
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Background 


•  In  May  2007,  ONDCP  issued  a  drug  control  accounting 
circular.  Section  7  of  the  circular  requires  that  National  Drug 
Control  Program  agencies,  including  DOD,  submit  to  the 
Director  of  ONDCP  a  performance  summary  report  including 
performance  measures,  targets,  and  results. 

•  ONDCP  cited  21  U.S.  Code  §  1703(d)(7),  §  1705(c)(4),  § 
1705  (a)(2)(xi)  as  the  legal  authorities  that  require  ONDCP  to 
monitor  the  implementation  of  the  National  Drug  Control 
Strategy,  develop  a  performance  measurement  system,  and 
assess  the  effectiveness  of  achieving  federal  National  Drug 
Control  Strategy  goals. 
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Background 


•  DASD-CN  reports  that  it  is  the  focal  point  for  DOD  counternarcotics 
efforts  and  that  it  will  ensure  the  department  develops  and 
implements  a  focused  counternarcotics  program  with  clear  priorities 
and  measured  results. 


•  Programs,  Resources,  and  Assessments,  a  division  within  DASD- 
CN,  is  the  lead  office  for  the  development  of  counternarcotics 
resources  and  plans.  Among  other  activities,  this  office 

•  directs  and  manages  the  planning,  programming,  and 
budgeting  system  of  the  DOD  counternarcotics  program,  and 

•  collects  quarterly  operational  statistics  to  evaluate  the 
effectiveness  of  DOD’s  contribution  to  meeting  the  goals  of  the 
National  Drug  Control  Strategy. 
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DOD  Has  Developed  Performance  Measures 
for  Its  Counternarcotics  Activities 

•  In  response  to  ONDCP  guidance,  DOD  developed  and  submitted  to 
ONDCP  a  set  of  performance  measures  for  its  fiscal  year  2007 
counternarcotics  activities.  Subsequently,  DOD  revised  its  performance 
measures  for  fiscal  years  2008  and  2009  and  submitted  a  report  on  these 
measures  to  ONDCP.2 

•  DOD  established  a  database  to  collect  performance  information  from  its 
components,  including  performance  measures,  targets,  and  results.  In 
June  2008,  DOD  issued  guidance  with  instructions  for  entering 
counternarcotics  performance  information  into  the  database. 

•  In  fiscal  year  2009,  DOD  reported  it  had  285  performance  measures  for  its 
counternarcotics  activities.  Of  those,  there  were  171  unclassified 
performance  measures  related  to  DOD’s  mission  of  supporting  law 
enforcement  agencies. 


According  to  ONDCP,  DOD’s  reports  for  fiscal  years  2007,  2008,  and  2009  did  not  fulfill  the  requirements  of  ONDCP’s 
drug  control  accounting  circular  because  the  reports  were  not  authenticated  by  the  DOD-IG. 
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GAO  Has  Identified  Nine  Key  Attributes  of 
Successful  Performance  Measures 


Table  1 :  GAO’s  Key  Attributes  of  Successful  Performance  Measures 


Attributes 

Definitions 

Potentially  adverse  effects  of  not  meeting  attribute 

Linkage 

Measure  is  aligned  with  division  and  agencywide  goals 
and  mission  and  clearly  communicated  throughout  the 
organization. 

Behaviors  and  incentives  created  by  measures  do  not 
support  achieving  division  or  agencywide  goals  or 
mission. 

Clarity 

Measure  is  clearly  stated,  and  the  name  and  definition 
are  consistent  with  the  methodology  used  to  calculate  it. 

Data  could  be  confusing  and  misleading  to  users. 

Measurable 

target 

Measure  has  a  numerical  goal. 

Cannot  tell  whether  performance  is  meeting 
expectations. 

Objectivity 

Measure  is  reasonably  free  from  significant  bias  or 
manipulation. 

Performance  assessments  may  be  systematically 
over-  or  understated. 

Reliability 

Measure  produces  the  same  result  under  similar 
conditions. 

Reported  performance  data  is  inconsistent  and  adds 
uncertainty. 

Core  program 
activities 

Measures  cover  the  activities  that  an  entity  is  expected 
to  perform  to  support  the  intent  of  the  program. 

Not  enough  information  available  in  core  program 
areas  to  managers  and  stakeholders. 

Limited 

overlap 

Measure  should  provide  new  information  beyond  that 
provided  by  other  measures. 

Manager  may  have  to  sort  through  redundant,  costly 
information  that  does  not  add  value. 

Balance 

Balance  exists  when  a  suite  of  measures  ensures  that  an 
organization’s  various  priorities  are  covered. 

Lack  of  balance  could  create  skewed  incentives  when 
measures  overemphasize  some  goals. 

Government¬ 
wide  priorities 

Each  measure  should  cover  a  priority  such  as  quality, 
timeliness,  and  cost  of  service. 

A  program's  overall  success  is  at  risk  if  all  priorities 
are  not  addressed. 

Source:  GAO-03-143. 
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DOD’s  Performance  Measures  Lack  Key 
Attributes  of  Successful  Measures 

•  DOD  and  ONDCP  officials  acknowledged  that  there  are 
weaknesses  in  the  performance  measures  in  DOD’s 
counternarcotics  database. 


•  GAO’s  analysis  of  a  sample  of  DOD’s  unclassified  fiscal  year  2009 
performance  measures  shows  that  key  attributes  were  not  always 
present  in  the  measures. 

•  Two  of  the  attributes,  balance  and  limited  overlap,  did  not 
appear  to  be  present  in  the  set  of  measures. 

•  The  other  seven  attributes  of  successful  performance  measures 
were  present  to  varying  degrees  (see  fig.  1). 

•  Our  preliminary  observations  suggest  that  DOD’s  classified  fiscal 
year  2009  performance  measures  may  have  similar  weaknesses. 
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DOD’s  Performance  Measures  Lack  Key 
Attributes  of  Successful  Measures 


Figure  1 :  DOD’s  Fiscal  Year  2009  Unclassified  Counternarcotics  Performance  Measures  Relative  to  Seven 
Attributes  of  Successful  Performance  Measures 
Key  attributes  of  performance  measures 


Government-wide  priorities 


Measurable  target 


Core  program  activities  | 

0  10  20  30  40  50  60  70  80  90  100 

Percentage  of  DOD  counternarcotics  performance  measures  that  exhibited  key  attributes 

Source:  GAO  analyses  of  DOO  performance  measures. 

Note:  GAO  randomly  sampled  71  of  DOD's  1 71  counternarcotics  performance  measures  for  fiscal  year  2009  that  were  not  classified  and  were 

associated  with  DOD’s  goals  of  detection  and  monitoring,  sharing  information,  and  building  capacity  of  partner  nations.  The  resulting  estimates,  1  3 
reported  above,  are  subject  to  a  maximum  margin  of  error  of  +/- 1 0  percentage  points. 
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DOD  Is  Working  To  Improve  Its  Performance 
Measures,  but  Some  Challenges  Remain 


•  DOD  officials  have  acknowledged  that  the  department’s 
counternarcotics  performance  measures  need  to  be  revised.  DOD 
hired  a  contractor  to  assist  it  in  revising  its  guidance  for  the 
development  of  counternarcotics  performance  measures.  Revised 
guidance,  dated  January  25,  2010,  exists  but  is  not  yet  finalized. 

•  CENTCOM  and  SOUTHCOM  officials  told  us  that  the  revised 
guidance  provides  better  direction  than  previous  guidance. 

•  We  plan  to  comment  on  DOD’s  revised  guidance  in  our  summer 
2010  product. 
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DOD  Is  Working  To  Improve  Its  Performance 
Measures,  but  Some  Challenges  Remain 


•  DOD  officials,  however,  noted  that  the  department  still  faces 
challenges  measuring  the  performance  of  its  counternarcotics 
activities.  These  challenges  include  the  following: 

•  creating  outcome-based  performance  measures  given  DOD’s 
supporting  role  in  counternarcotics  activities, 

•  ensuring  adequate  resources,  such  as  expertise  in  performance 
management,  are  available  to  develop  performance  measures, 
and 

•  ensuring  data  used  to  measure  DOD  performance  is  reliable. 
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GAO  Has  Identified  Use  of  Performance 
Information  As  Key  To  Improving  Management 

Figure  2:  Uses  of  Performance  Information 


Uses  of  Performance  Data 


Improved 

management 

decision 

making 


Source:  GAO. _ 

Note:  GAO  has  also  previously  reported  that  recognizing  and  rewarding  individual  performance  is  a  use  of  performance  data.  -\  0 

However,  this  use  was  not  applicable  to  DOD’s  counternaroctics  performance  measures,  as  they  relate  only  to  program  activities. 
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Preliminary  Observations 


i  G  AO 

»—  Accountability  •  Integrity  *  Reliability 


DOD’s  Use  of  Performance  Information  to 
Support  Its  Counternarcotics  Mission 


•  Identify  problems  and  take  corrective  action:  CENTCOM,  SOUTHCOM,  and 
JIATF-S  officials  told  us  they  rarely  use  the  DASD-CN  database  to  identify 
performance  problems  or  manage  counternarcotics  activities.  Rather,  they  tend  to 
manage  programs  using  information  not  submitted  to  the  DASD-CN  database.  For 
example,  CENTCOM  officials  told  us  information  obtained  in  weekly  program 
meetings,  not  data  sent  to  the  DASD-CN  database,  is  most  often  used  to  help 
them  identify  problems  and  make  program  adjustments.  Further,  it  is  unclear  the 
extent  to  which  DASD-CN  uses  information  in  the  database  to  oversee 
counternarcotics  activities. 


•  Develop  strategy  and  allocate  resources:  DASD-CN  officials  told  us  that  they  do 
not  currently  link  performance  information  directly  to  budget  allocation  decisions. 


•  Identify  and  share  effective  approaches:  While  DOD  creates  reports  for  ONDCP, 
DASD-CN  officials  told  us  they  do  not  produce  additional  reports  for  the 
department. 
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GAO 

Accountability  •  Integrity  •  Reliability 


Agency  Comments  and  Our  Evaluation 


•  We  provided  drafts  of  this  report  to  DOD  and  ONDCP  for 
comment.  DOD  provided  no  comments.  ONDCP  provided 
technical  comments,  which  we  have  incorporated  where 
appropriate. 
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This  is  a  work  of  the  U  .S .  government  and  is  not  subject  to  copyright  protection  in  the 
United  States.  The  published  product  may  be  reproduced  and  distributed  in  its  entirety 
without  further  permission  from  GAO.  However,  because  this  work  may  contain 
copyrighted  images  or  other  material,  permission  from  the  copyright  holder  may  be 
necessary  if  you  wish  to  reproduce  this  material  separately. 


GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  Web  site  newly  released  reports,  testimony,  and 
correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted  products, 
go  to  www.gao.gov  and  select  “E-mail  Updates.” 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO’s  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO’s  Web  site, 
http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Ralph  Dawn,  Managing  Director,  dawnr@gao.gov,  (202)  512-4400 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7125 
Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngcl@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 

